OPINION 


BATTLE  LABS: 
TOOLS  AND  SCOPE 

Julian  Cothran 


The  Battle  Lab  is  a  tool  for  the  rapid  insertion  of  new  technology  into  weap¬ 
ons  systems  and  for  the  early  evaluation  of  potential  military  components 
and  experimental  systems.  It  can  yield  cost  savings  to  project  managers  and 
system  users.  It  is  multi-faceted,  meeting  such  diverse  requirements  of  the 
acquisition  process  as  the  engineering  test  beds  used  by  the  project  man¬ 
ager  and  the  simulations  used  by  commanders,  planners  and  others  for 
wargaming.  This  paper  describes  the  desired  integration  of  battle  labs  with 
test  beds,  and  how  test  beds  produce:  a)  the  required  fidelity  of  input  for 
Battle  Lab  demonstrations;  and,  b)  experiments  with  evolving  technological 
advancements. 


According  to  General  Fredrick 
M.  Franks,  Jr.,  former  com¬ 
mander  of  the  U.S.  Army 
Training  and  Doctrine  Command 
(TRADOC): 

(W)hat  we  wanted  to  do  in 
TRADOC  was  provide  ourselves 
a  means — given  resource  con¬ 
straints — to  take  emerging  ideas 
from  recent  battlefield  experi¬ 
ences  such  as  Just  Cause  and 
Desert  Storm  and  continue  to  ex¬ 
periment  with  those  ideas  and  with 
technology  insertions  that  could 
be  applied  to  furthering  our  war¬ 
fighting  capabilities  using  simula¬ 
tions  as  well  as  some  actual  proto¬ 
type  (hardware)  systems  tied  in 


with  the  simulations  (Roos  and 
Franks,  1992). 

This  can  be  accomplished  by  “net¬ 
working  simulators  that  offer  a  safe, 
cost  effective  environment  augmenting 
live  field  exercises;  one  in  which  we  can 
afford  to  exercise  all  the  components 
of  today’s  combined  arms  teams,”  ac¬ 
cording  to  George  T.  Singley,  III 
(Singley,  1993).  Singley  adds: 

(M)aterial  developers  will  shorten 
acquisition  time  while  reducing 
both  costs  and  development  risks 
by  employing  Distributed  Interac¬ 
tive  Simulation  (DIS)  during  con¬ 
cept  definition,  concept  explora¬ 
tion,  design,  MANPRINT  assess- 
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ments  and  prototyping. 

Simulation  also  allows  for  quicker, 
more  effective  trade-off  studies.  The 
result  is  clearer  requirements  in  less 
time  and  at  lower  cost  (Franks  and 
Ross,  1993;  Slear,  1992). 

Gen.  Franks  and  General  Jimmy  D. 
Ross,  former  commander  of  the  Army 
Materiel  Command  (AMC),  agreed 
that: 

Battle  Labs’  requirements  for  rapid 
insertion  of  new  technologies  into  sys¬ 
tems  via  components  and  experimen¬ 
tal  systems  will  be  tested  iteratively, 
demonstrated  and  evaluated  for  mili¬ 
tary  value.  To  a  much  greater  degree 
than  in  the  past,  this  process  is  based 
on  simulation  of  both  the  physical  sys¬ 
tem  and  its  battlefield  performance. 
Battle  Labs  provide  a  means  for  the 
Army’s  systematical  examination  of 
war-fighting  ideas  and  evaluation  of  the 
options  offered  by  new  technical  capa¬ 
bilities  (Franks  and  Ross,  1993). 

They  went  on  to  say  that, 

(T)he  objective  of  each  Battle  Lab 
is  to  determine  the  potential  mili¬ 
tary  value  offered  by  a  new  capa¬ 
bility  as  early  as  possible.  Products 
of  these  efforts  typically  are  soft¬ 
ware  models  or  early  stage  ‘aus¬ 


tere  prototypes’  such  as  ‘bread¬ 
boards’  or  ‘brassboards’  without 
the  full  functionality  of  complete 
fieldable  systems  or  components. 
Testing  is  likely  informal  and  may 
involve  an  iterative  model-fix- 
model  or  test-fix-test  cycle. 

This  means  of  virtual  prototyping  not 
only  facilitates  concurrent  engineering 
but  also  encourages  continuous,  com¬ 
prehensive  evaluation  by  the  combat 
development,  material  development, 
and  test  and  evaluation  communities  at 
the  beginning  of  the  acquisition  pro¬ 
cess — ^when  the  weapon  system  is  be¬ 
ing  designed  to  reduce  the  time  and 
cost  of  the  acquisition  cycle  (Ross, 
1993;  Singley,  1993). 

In  summary,  the  PM  must  develop 
test  bed  tools  and  integrate  his  efforts 
with  the  Battle  Labs  if  he  is  to  demon¬ 
strate  system  capabilities  that  are  not 
only  measurable,  but  also  result  in  the 
high  fidelity  simulations  that  will 
streamline  the  acquisition  process. 
Battle  Labs,  the  Louisiana  Maneuvers 
(LAM),  and  the  methodology  of  DIS 
combine  nicely  to  point  the  way,  but  the 
proof  is  in  the  implementation.  Prob¬ 
lems  encountered  in  implementing  the 
concept  and  methodologies  of  simula¬ 
tion  frequently  involve  the  mispercep¬ 
tions  of  decision  makers.  Among  these 
is  the  widely  shared  misperception  that 


Julian  Cothran  was  the  Chief  Engineer,  Forward  Area  Air  Defense  Project  Office,  U.S. 
Army  Missile  Command,  Redstone  Arsenal,  Alabama,  from  1986  through  March  1995.  He 
is  a  graduate  of  PMC  94-1,  Defense  Systems  Management  College  (DSMC). 


52 


Battle  Labs 


testing  must  deliver  sensational,  ‘crash 
and  burn’  results  to  be  deemed  effec¬ 
tive  by  the  public.  This  erroneous  ex¬ 
pectation  must  give  way  to  a  desire  for 
the  in-depth,  structured  testing  and 
analyses  performed  in  a  test  bed  and 
Battle  Lab  environment.  That  new  en¬ 
vironment  truly  provides  the  qualitative 
and  quantitative  data  about  a  weapon 
system’s  added  value  that  will  support 
program  and  system  decisions. 

Another  misperception  lies  within 
the  systems  engineering  process.  Al¬ 
though  the  steps  in  the  process  are 
good,  how  and  when  these  steps  are 
executed  is  not  unalterable,  and  the 
perception  that  they  are  is  mistaken. 
This  is  pivotal  to  the  success  of  Battle 
Labs. 

To  implement  simulation  properly 
requires  a  teaming  of  the  user  (or  com¬ 
bat  developer)  and  the  Project  Man¬ 
ager  (or  material  developer).  The  aim 
of  their  combined  efforts  reflects  a  con¬ 
current  engineering  philosophy  that 
provides  direct  feedback  into  the 
weapon  system  development  cycle.  The 
goal  is  to  use  modeling  and  simulation 
to  test,  evaluate,  and  further  amplify 
any  number  of  factors  in  that  cycle. 
Among  these:  Operational  Require¬ 
ment  Document  (ORD)  requirements, 
smart  technology  insertion,  comparison 
of  alternative  evolutionary  concepts, 
predictions  of  the  system’s  functional 
and  operational  performance,  design 
and  development  of  new  devices  and 
algorithms,  system  integration,  system 
software  support,  command  and  con¬ 
trol,  best  doctrinal  way  to  fight  the  sys¬ 
tem,  and  MANPRINT  issues.  These  as¬ 
sessment  and  development  needs  are 
not  new;  however,  that  they  are  ob¬ 


tained  as  a  joint  team  effort  is  new. 

The  Battle  Labs  concept,  with  speci¬ 
fied  centers  controlled  by  the  combat 
developer,  is  well-understood.  Unfor¬ 
tunately,  the  contribution  of  the  test 
bed  to  the  Project  Manager’s  team  is 
less  visible,  as  is  the  interplay  of  test  bed 
data  used  by 

combat  devel-  ...the  widely-shared 
opers  and  ma-  misper<eptien  that 
terial  develop-  testing  must  deliver 
ers.  Neverthe-  sensatienal,  '«rash  and 
less,  a  fusion  of  burn'  results... 
the  test  beds 

and  battle  labs,  providing  end-to-end 
simulations  and  simulators,  would  fos¬ 
ter  rapid  prototyping  through  ‘hard- 
ware-in-the-loop’  (HWIL).  It  would 
also  combine,  in  a  DIS  synthetic  envi¬ 
ronment,  the  domains  of  research,  de¬ 
velopment  and  acquisition  (RD&A), 
military  operations,  and  training  (see 
Figure  1). 

Project  managers  own  the  detailed 
simulations  (or  test  beds)  that  provide 
accurate  weapon  system  performance 
data  to  wargaming  models.  These  com¬ 
plex  test  beds  place  soldiers  in  detailed 
simulations  of  hardware  prototypes  and 
new  system  software  to  assess  the 
weapon’s  warfighting  ‘value  added.’ 
Through  DIS,  test  beds  enable  a  new 
weapon  system,  or  a  new  configuration 
of  an  old  weapon  system,  to  interact  in 
a  war  game  in  real  time.  The  combat 
developer  is  given  access  to  the  simu¬ 
lation  at  his  home  station.  This  is  the 
Battle  Lab  concept  enabled  through  a 
teaming  of  users,  PMs,  contractors,  and 
developers. 

Whether  test  beds  support  the  re¬ 
quired  evaluation  areas  and  address  the 
widest  scope  of  issues  (while  remain- 
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A  Time  and  Space  Coherent  Representation  of  a  Battiefieid  Environment 
Measured  in  Terms  of  Human  Perception  and  Behavior 
of  Those  Interacting  in  the  Environment 


^  TEST  &  EVAL  ^ 

(i 

r. 


MATERIEL  EVAL 


COMBAT  EVAL 


Qbainin^va^ 


RD&A 

DOMAIN 


Figure  1.  DIS  SynthetU  Environment 


ing  flexible  and  comparatively  inexpen¬ 
sive)  should  be  asked  in  determining 
exactly  what  types  and  combinations  of 
simulations,  test  beds,  and  tests  are 
needed.  This  evaluation  is  illustrated  in 
Figures  2  and  3. 

Evaluation  Areas 


The  next  step  is  to  assess  whether  the 
detail  and  scope  of  the  evaluation  meth¬ 
odology  will  support  technical  require¬ 
ments  generation  and  evaluation,  op¬ 
erational  requirements,  and  overall  re¬ 
quirements  (e.g.,  force  modernization). 
The  assessment  of  these  applications  is 


shown  in  Figure  4,  as  applied  to  the 
STINGER/AVENGER,  the  Forward 
Area  Air  Defense  (FAAD)  Project  Of¬ 
fice,  and  the  Weapon  System  Manage¬ 
ment  Directorate  (WSMD).  Next  we 
assess  the  scope  and  detail  of  the  tools 
[missile  simulation  (HWIL);  weapon 
system  fire  unit  simulation  (HWIL); 
Software  in  the  Loop,  (or  SWIL);  and 
Man  in  the  Loop,  (or  MIL),  and  the 
battlefield  models],  and  how  these  tools 
interact  with  each  other  within  the  DIS 
virtual  network.  This  is  illustrated  in 
Figure  5  as  an  integrated  evaluation 
and  Test  Evaluation  Master  Plan 
(TEMP)  asset. 
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Figure  2.  System  Evaluation  Methodology  Trade-Off 


SYS.  COMPONENT  COMPONENT  SYSTEM  SYS. 

ARCH.  DESIGN  EVAL.  INTEG.  EVAL. 
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LEGEND: 
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SIM -SIMULATION 

ARCH  -  ARCHITECTURE 

EX  -  EXAMPLE 

DEM  -  DEMONSTRATION 

EVAL  -  EVALUATION 

MSL- MISSILE 

VAL  -  VALUATION 

INTEG -INTEGRATION 

DOF  -  DEGREE  OF  FREEDOM 

Figure  3.  Evaluation  Areas 
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Figure  4.  FAADS  Simulation  Test  Bed, 
System  Evaluation  Methodologies/Appli<ations 
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Figure  5.  Scope,  Detail,  and  Interaction 
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Figure  6.  Simulation  Hierarthy  (PM's  Tool  Kit) 


After  determining  the  detail  and 
scope  required  of  the  test  bed,  simula¬ 
tions  and  models,  an  ordering  by  type 
and  function  needs  to  be  performed  to 
produce  a  simulation  hierarchy.  This 


type  of  hierarchy  is  shown  in  Figure  6 
for  the  FAAD  PM  and  WSMD.  The 
next  assessment  determines  how  and 
when  the  various  tools  will  be  needed, 
and  how  they  should  connect  (see  Fig- 
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f  FOCUS  ON  ANALYSIS,  TEST  ^ 
WITH  SOME  TRAINING 


Figure  7.  Simulation  Evolution/Life  Cy<le 


ure  7).  Within  this  evolution,  the  simu¬ 
lated  weapon  system  prototypes  are 
evaluated  by  soldiers.  This  process  of 
virtual  prototyping  produces  the  ben¬ 
efits  seen  in  Figure  8. 

The  Air  Defense  Program  Executive 
Officer  (PEO)  completed  an  initial  re¬ 
view  of  the  library  of  battlefield  mod¬ 
els  in  1989.  He  concluded  that  no  ex¬ 
isting  model  could  provide  all  of  the 
features  needed  and  desired  for  analy¬ 
ses  of  Forward  Area  Air  Defense 
(FAAD)  systems.  Instead,  it  would  be 
necessary  to  use  several  models  in  sup¬ 
port  of  system  performance  assess¬ 
ments,  tactics,  and  doctrine  analyses. 
The  survey  identified  minimum  re¬ 
quirements  for  models  and  defined  cri¬ 
teria  for  evaluating  and  comparing 


models.  A  subsequent  evaluation  of 
each  model’s  applicability  and  utility  for 
analyzing  FAAD  issues  was  also  con¬ 
ducted.  This  revealed  that  the  models 
would  have  to  be  capable  of  support¬ 
ing  battalion-sized  or  larger  units  in  an 
asymmetric  play  of  forces  (e.g..  Blue 
tactics  by  Blue,  Red  tactics  by  Red). 
The  models  would  also  have  to  be  in 
use  at  present  in  the  simulation  com¬ 
munity. 

The  CASTFOREM,  JANUS(T), 
and  VIC  models  were  chosen;  together, 
the  three  models  satisfied  the  battle¬ 
field  integration  issues.  Many  of  the 
detailed  outputs  from  the  interactive 
JANUS(T)  could  be  fed  into 
CASTFOREM.  Similarly,  some  of  the 
battalion  and  brigade-level  results  from 
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Figure  8.  Virtual  Prototype  Simulation  ■  Life  Cycle 


CASTFOREM  could  be  used  as  input 
for  the  corps  and  division-level  VIC 
simulation  (Air  Defense  PEO,  1989). 
Yet  all  three  had  major  drawbacks: 
They  assumed  perfect  identification  of 
friend  or  foe  (IFF);  they  modeled  com¬ 
mand  and  control  logic  through  deci¬ 
sion  tables  only,  thus  not  allowing  for 
assessment  of  a  C3I  capability  on  the 
battlefield;  they  lacked  detail  in  the  play 
of  fixed-wing  aircraft;  they  excluded 
fratricide;  they  allowed  no  explicit  elec¬ 
tronic  warfare  play;  and  they  used  un¬ 
changing  weather  parameters.  In  addi¬ 
tion,  JANUS(T)  provided  only  a  very 
coarse  level  of  modeling  for  a  fire  unit 


by  assuming  perfect  targeting  by  the 
threat  (Red)  aircraft,  an  ‘a  priori’  know¬ 
ledge  of  Blue’s  location  by  Red  aircraft, 
and  visual  identification  ranges  for  Blue 
forces  applicable  to  tanks  rather  than 
the  detection,  recognition,  and  identi¬ 
fication  ranges  common  to  sensors  in 
Air  Defense  units.  Nevertheless,  these 
limitations  of  the  models  make  a  test 
bed  attractive  since  their  outputs,  when 
inserted  into  the  VIC  battle,  can  easily 
provide  the  correct  inputs  for  a 
CASTFOREM  or  VIC  model,  or  any 
upgrade  of  these  with  higher  resolution 
and  fidelity. 
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Figure  9.  Test  Bed  ApplUatiens 


The  Utility  Of  Implementation 


Within  the  constraints  of  the  preced¬ 
ing  section,  the  test  bed  is  an  analysis 
tool  emulating  the  weapon  system  and 
used  to  conduct  experiments,  studies 
and  analyses  to  support:  a)  predictions 
of  the  system’s  functional  and  opera¬ 
tional  performance;  b)  comparison  of 
alternative  evolutionary  concepts;  c) 
design  and  development  of  new  con¬ 
cepts,  devices  and  algorithms;  d)  sys¬ 
tem  integration;  and  e)  maintenance 
and  support  of  system  software 
(AVENGER  Project  Office,  1992)  (see 
Figure  9). 

A  wide  variety  of  concept  and  con¬ 
figuration  trades-offs  are  necessary  in 
any  system’s  evolutionary  development. 


This  use  of  test  beds  is  manifested  at 
several  levels.  At  one  level,  the  test  bed 
allows  investigation  into  alternative 
structures  for  weapon  systems  or  rela¬ 
tionships  between  system  components 
(e.g.,  the  effects  of  system  mod¬ 
ularization,  element  intercommunica¬ 
tion,  and  centralization  of  decision 
making).  Another  level  of  test  bed  use 
is  the  selection  of  alternative  weapon 
system  elements  (i.e.,  technology  inser¬ 
tion)  based  on  comparisons  of  their  ef¬ 
fectiveness.  A  third  level  of  test  bed 
utility  lies  in  measuring  variations  in  a 
significant  component’s  characteristics 
and  how  they  impact  the  effectiveness 
of  the  full  system.  These  three  levels  of 
analyses  provide  the  basis  for  informed 
decisions  on  trade-offs. 
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Test  beds  also  support  eomponent 
design  and  development.  The  test  bed 
is  used  to  shake  down  preliminary  de¬ 
signs  (i.e.,  analytieal  models),  evaluat¬ 
ing  them  in  the  context  of  weapon  sys¬ 
tem  objects  and  functions.  This  enables 
the  subsystem  to  be  studied  in  a  con¬ 
trolled  but  realistic  operational  envi¬ 
ronment  for  which  the  design  variables 
serve  as  study  parameters.  It  also  allows 
other  elements  of  the  system  to  influ¬ 
ence  modification  and  evaluation  of  the 
design,  as  well  as  permitting  observa¬ 
tion  of  the  effects  of  design  parameters 
on  system  performance. 

Design  and  development  of  compo¬ 
nents  and  subsystems  are  brought  to¬ 
gether  in  system  integration.  The  test 
bed  has  tremendous  utility  for  reduc¬ 
ing  the  high  risk  in  this  area.  System 
integration  issues  explored  on  the  test 
bed  include  functional  or  operational 
coordination,  completeness,  and  integ¬ 
rity;  system  interface  validation;  data 
fusion;  and  the  cooperative  operation 
of  system  elements.  The  test  bed  may 
also  serve  to  identify  and  quantify  any 
problems  in  functional  or  data  interface 
and  to  investigate  alternative  solutions 
for  such  problems,  and  also  serve  to 
support  experiments  or  demonstrations 
of  system  integration  concepts. 

Performance  assessment  of  a  weapon 
system  is  another  use  of  the  test  bed, 
as  is  validation  of  engagement  simula¬ 
tions  or  wargaming  models.  The  test 
bed  can  generate  data  invaluable  in 


validating  engagement  models  by  dem¬ 
onstrating  the  system’s  fully  integrated 
operational  functions  under  full  en¬ 
gagement  scenarios.  Using  the  test  bed 
to  predict  the  results  of  a  weapon 
system’s  field  and  firing  tests  supports 
pre-test  planning  and  post-test  analy¬ 
ses,  reduces  the  amount  of  real  world 
testing  required,  saves  time  and  money, 
and  provides  results  that  are  more  con¬ 
structive  and  defensible. 

In  today’s  software  driven  weapon 
systems,  the  test  bed  is  a  necessary 
complement  to  the  normal  software 
development  environment.  The  test 
bed  provides  the  complex  and  realistic 
stimuli  and  operational  states  necessary 
to  determine  the  adequacy  of  the 
weapon  system’s  operational,  imbed¬ 
ded  software. 

Summary 


The  Test  Bed  and  Tools  for  DIS  are 
ready  and  functional.  Integration  and 
consolidation  of  efforts  to  utilize  these 
tools  must  be  continued.  Many  re¬ 
sources  and  simulations  are  untapped 
that  can  help  the  Project  Managers  and 
the  user.  The  Program  Executive  Of¬ 
ficer  and  TRADOC  User  communities 
need  to  synchronize  their  efforts  to  cre¬ 
ate  cost  effective  weapons  systems  and 
system  improvements  through  robust 
simulations. 
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